XCS Algorithms for a Linear Combination of Discounted and Undiscounted Reward Markovian Decision Processes

RÉSUMÉ : Plusieurs études ont montré que combiner certains prédicteurs ensemble peut améliorer la justesse de la prédiction dans certains domaines comme la psychologie, les statistiques ou les sciences du management. Toutefois, aucune de ces études n'ont testé la combinaison de techniques d'apprentissage par renforcement. Notre étude vise à développer un algorithme basé sur deux algorithmes qui sont des formes approximatives d'apprentissage par renforcement répétés dans XCS. Cet algorithme, MIXCS, est une combinaison des techniques de Q-learning et de R-learning pour calculer la combinaison linéaire du payoff résultant des actions de l'agent, et aussi la correspondance entre la prédiction au niveau du système et la valeur réelle des actions de l'agent. MIXCS fait une prévision du payoff espéré pour chacune des actions disponibles pour l'agent. Nous avons testé MIXCS dans deux environnements à deux dimensions, Environment1 et Environment2, qui reproduisent les actions possibles dans un marché financier (acheter, vendre, ne rien faire) pour évaluer les performances d'un agent qui veut obtenir un profit espéré. Nous avons calculé le payoff optimal moyen dans nos deux environnements et avons comparé avec les résultats obtenus par MIXCS. Nous avons obtenu deux résultats. En premier, les résultats de MIXCS sont semblables au payoff optimal moyen pour Environments1, mais pas pour Environment2. Deuxièmement, l'agent obtient le payoff optimal moyen quand il prend l'action "vendre" dans les deux environnements.----------ABSTRACT : Many studies have shown that combining individual predictors improved the accuracy of predictions in different domains such as psychology, statistics and management sciences. However, these studies have not tested the combination of reinforcement learning techniques. This study aims to develop an algorithm based on two iterative approximate forms of reinforcement learning algorithm in XCS. This algorithm, named MIXCS, is a combination of Q-learning and R-learning techniques to compute the linear combination payoff and the correspondence between the system prediction and the action value. As such, MIXCS predicts the payoff to be expected for each possible action. We test MIXCS in two two-dimensional grids called Environment1 and Environment2, which represent financial markets actions of buying, selling and holding to evaluate the performance of an agent as a trader to gain the desired profit. We calculate the optimum average payoff to predict the value of the next movement in both Environment1 and Environment2 and compare the results with those obtained with MIXCS. The results show that the performance of MIXCS is close to optimum average reward in Environment1, but not in Environment2. Also, the agent reaches the maximum reward by taking selling actions in both Environments

Synthesis and Preformulation Studies of KTTKS and PAL-KTTKS as Anti-Wrinkle Peptides

Introduction: Skin aging is a complicated process which is one of the major issues in the field of dermatology and cosmetic products. Peptides are one of the novel ingredients included in the anti-aging formulations. KTTKS (Lys-Thr-Thr-Lys-Ser) and its derivative, PAL-KTTKS (Palmitic acid-KTTKS), have attracted a lot of attention in arresting or delaying skin aging, but unfortunately, there are almost no preformulation studies available about them. Methods and Results:Both peptides were synthesized by solid phase peptide synthesis and identified by Mass spectroscopy technique. UV absorption ability, percentage of crystallinity, melting point, decomposition temperature and thermal behavior of both peptides were analyzed by UV spectroscopy, XRD, TGA and DSC techniques respectively. Partition coefficient was also determined by ACD/chemsketch software. In addition stability studies for the aqueous solution of KTTKS were performed at 32 and 37 ℃.  The results of UV spectroscopy showsthe wavelength of maximum absorbance of both peptides is in the vacuum UV range. Based on the results of melting point and TGA apparatuses, KTTKS and PAL- KTTKS decompose at about 154 ℃ and 112 ℃ respectively and there is no melting point for them before decomposition. The results of DSC thermogramsindicate an endothermic peak at the temperature below 60 ℃ for both peptides which is probably due to intrinsic structural rearrangement or evaporation of volatile solvents. Crystallinity percentage for KTTKS and PAL-KTTKS are 62% and 32% respectively. cLogp of KTTKS is -3.27 and cLogp of PAL-KTTKS is 3.32. Conclusions: The results of this investigation can be employed for the formulation of these peptides for TTD

Quantitative assessment of wound healing using high-frequency ultrasound image analysis

Purpose: We aimed to develop a method for quantitative assessment of wound healing in ulcerated diabetic feet. Methods: High‐frequency ultrasound (HFU) images of 30 wounds were acquired in a controlled environment on post‐debridement days 7, 14, 21, and 28. Meaningful features portraying changes in structure and intensity of echoes during healing were extracted from the images, their relevance and discriminatory power being verified by analysis of variance. Relative analysis of tissue healing was conducted by developing a features‐based healing function, optimised using the pattern‐search method. Its performance was investigated through leave‐one‐out cross‐validation technique and reconfirmed using principal component analysis. Results: The constructed healing function could depict tissue changes during healing with 87.8% accuracy. The first principal component derived from the extracted features demonstrated similar pattern to the constructed healing function, accounting for 86.3% of the data variance. Conclusion: The developed wound analysis technique could be a viable tool in quantitative assessment of diabetic foot ulcers during healing

Information Loss Due to the Data Reduction of Sample Data from Discrete Distributions

In this paper, we study the information lost when a real-valued statistic is used to reduce or summarize sample data from a discrete random variable with a one-dimensional parameter. We compare the probability that a random sample gives a particular data set to the probability of the statistic’s value for this data set. We focus on sufficient statistics for the parameter of interest and develop a general formula independent of the parameter for the Shannon information lost when a data sample is reduced to such a summary statistic. We also develop a measure of entropy for this lost information that depends only on the real-valued statistic but neither the parameter nor the data. Our approach would also work for non-sufficient statistics, but the lost information and associated entropy would involve the parameter. The method is applied to three well-known discrete distributions to illustrate its implementation

Optimization of producing oil and meal from canola seeds using microwave − pulsed electric field pretreatment

In this study, optimization of the extraction of canola seeds oil was investigated using microwave-pulsed electric field seeds pretreatment (MW-PEF) with different MW times (0 to 200 s) and PEF intensities (0 to 5 kV/cm). The seeds oil was then extracted using screw press with different speeds (11 to 57 rpm). Oil extraction efficiency, refractive index, peroxide and phenolic compounds of oil and meal protein were measured. Tocopherols content of the best sample was also measured. The results showed that the peroxide and phenolic compounds increased at higher time, intensity and speed. An increase in the MW time and PEF intensity at first led to an increase in the oil extraction efficiency and meal protein but then both parameters decreased. The efficiency of oil extraction and protein decreased at higher speeds. The refractive index of all samples was 1.475. Gamma tocopherol was predominate one in canola oil and applying the pretreatment led to an increase in the number of total tocopherols. Treating at 1.28 kV/cm for 140.5 s and 28.71 rpm was chosen as the optimum condition with high desirability (0.744)

Prevalence of preeclampsia and its maternal and fetal complications in women referring to Amiralmomenin Hospital of Zabol in 2014-2015

Background and Aim: Preeclampsia is one of the most important complications of pregnancy which complicates 5-8 percent of all pregnancies and is associated with increased maternal and fetal complications. This study aimed to determine the prevalence of preeclampsia and its fetal and maternal complications in pregnant women referring to the maternity ward of the Zabol-based Amiralmomenin Hospital. Materials and Methods: This is a retrospective, descriptive study conducted on the files of 2000 pregnant women referred to Amiralmomenin Hospital in Zabol. The data collection tools consisted of a two-part form that covered demographic and obstetric data as well as maternal and fetal complications of preeclampsia. Data were presented in descriptive statistics. Results: The prevalence of preeclampsia in Zabol was 6.5%. The frequency of the major maternal complications include liver dysfunction 13.1%, renal disorders 3.1%, transfusion 4.6%, thrombocytopenia 2.3%, visual impairment 2.3%, stillbirth 0.8% and HELLP syndrome 0.8%. Fetal complications involve prematurity 29.2%, amniotic fluid meconial 12.3%, and Apgar score below 7 at birth 7.7%. Conclusion:  Given the prevalence of preeclampsia and its complications for the mother and the fetus, proper care during pregnancy should be provided in order for early detection and prevention of adverse effects